Comment: Understanding Simpson’s Paradox
نویسنده
چکیده
I thank the editor, Ronald Christensen, for the opportunity to discuss this important topic and to comment on the article by Armistead. Simpson’s paradox is often presented as a compelling demonstration of why we need statistics education in our schools. It is a reminder of how easy it is to fall into a web of paradoxical conclusions when relying solely on intuition, unaided by rigorous statistical methods. In recent years, ironically, the paradox assumed an added dimension when educators began using it to demonstrate the limits of statistical methods, and why causal, rather than statistical considerations are necessary to avoid those paradoxical conclusions (Wasserman 2004; Arah 2008; Pearl 2009, pp. 173–182). My comments are divided into three parts. First, I will give a brief summary of the history of Simpson’s paradox and how it has been treated in the statistical literature in the past century. Next, I will ask what is required to declare the paradox “resolved,” and argue that modern understanding of causal inference has met those requirements. Finally, I will answer specific questions raised in Armistead’s article and show how the resolution of Simpson’s paradox can be taught for fun and progress.
منابع مشابه
Understanding Simpson’s Paradox
Simpson’s paradox is often presented as a compelling demonstration of why we need statistics education in our schools. It is a reminder of how easy it is to fall into a web of paradoxical conclusions when relying solely on intuition, unaided by rigorous statistical methods. In recent years, ironically, the paradox assumed an added dimension when educators began using it to demonstrate the limit...
متن کاملAveraging Gone Wrong: Using Time-Aware Analyses to Better Understand Behavior
Online communities provide a fertile ground for analyzing people’s behavior and improving our understanding of social processes. Because both people and communities change over time, we argue that analyses of these communities that take time into account will lead to deeper and more accurate results. Using Reddit as an example, we study the evolution of users based on comment and submission dat...
متن کاملComputational Social Scientist Beware: Simpson's Paradox in Behavioral Data
Observational data about human behavior is often heterogeneous, i.e., generated by subgroups within the population under study that vary in size and behavior. Heterogeneity predisposes analysis to Simpson’s paradox, whereby the trends observed in data that has been aggregated over the entire population may be substantially different from those of the underlying subgroups. I illustrate Simpson’s...
متن کاملHow Likely is Simpson's Paradox in Path Models?
Simpson’s paradox is a phenomenon arising from multivariate statistical analyses that often leads to paradoxical conclusions; in the field of e-collaboration as well as many other fields where multivariate methods are employed. We derive a general inequality for the occurrence of Simpson’s paradox in path models with or without latent variables. The inequality is then used to estimate the proba...
متن کاملThe Paradox of Intervening in Complex Adaptive Systems; Comment on “Using Complexity and Network Concepts to Inform Healthcare Knowledge Translation”
This commentary addresses two points raised by Kitson and colleagues’ article. First, increasing interest in applying the Complexity Theory lens in healthcare needs further systematic work to create some commonality between concepts used. Second, our need to adopt a better understanding of how these systems organise so we can change the systems overall behaviour, creates a paradox. We seek to m...
متن کامل